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Abstract 

Recent reports by government agencies have shown concern regarding the extent to which our 
nation’s workers lack the workplace skills necessary to meet the challenges of today’s workplace. 
This paper describes the Work Keys system, a program developed by American College Testing 
(ACT) to help improve the job skills of the workforce, and a project being conducted with the 
Oklahoma Department of Human Services (DHS) to examine the extent to which Work Keys 
assessment information can be used to address the human resources needs for three classifications 
of social workers. Job profiling, the job analysis component of the Work Keys system, was used 
to study all three jobs and establish content-related validity and skill standards. The assessment 
component of the system was used to examine the extent to which a sample of incumbents meets 
the skill levels set by subject matter experts. In the final stage of this project, performance data 
provided a basis to calculate statistical indices to address the criterion-related validity of the Work 
Keys system relative to the social worker jobs. Recommendations are given regarding how the 
Work Keys system can be used to address the human resources needs of the three social worker 
classifications studied in this project 
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Assessment Strategies: Validation of the Work Keys System 

for Social Workers 

Introduction to the Work Keys System 

In the past decade, concern has mounted that our nation’s workers, both current and future, 
lack the workplace skills necessary to meet the challenges of technological advances, organizational 
restructuring, and global economic competition. Increasingly, jobs require individuals to possess 
generic employability s kills that include problem-solving, communication, and personal skills. Yet 
nationwide studies, such as A Nation at Risk (National Commission on Excellence in Education, 
1983), and America 2000: An Educational Strategy (Secretary’s Commission on Achieving 
Necessary Skills, 1991), report that many people in the workforce lack these skills. This is 
particularly problematic because the majority of the people who will be in the workforce when it 
enters the 21st century are already on the job today (Auerbach, 1991). 

This is one of the challenges that prompted American College Testing (ACT) to expand its 
services and develop the Work Keys system as a means of improving generic workplace skills (i.e., 
those skills crucial to effective performance in most jobs). Work Keys is a national system 
designed to identify and improve workplace skills which serve employers, educational entities, and 
individual learners. The system is a multifunctional program of four interactive components: 
assessment, job profiling (job analysis), instructional support, and reporting. After considerable 
review, ACT, with the assistance of advisory panels comprised of employers, educators, and experts 
in employment and training requirements, selected twelve critical skills to form the basis of the 
Work Keys system: reading for information, applied mathematics, listening, writing, teamwork, 
applied technology, locating information, observation, speaking, motivation, learning, and managing 
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resources. To date, the Work Keys system is operational with the first eight employability skills 
cited above. Additional skill areas, selected from among these and others more recently identified, 
will be developed over time. 

Fundamental to the Work Keys system are the skill scales that measure both the generic 
employability skills required for specific jobs and those same employability skills demonstrated by 
individuals. These skill scales provide employers, educators, and individuals with a common 
language they can use to communicate information about skill qualifications and requirements. For 
example, employers and potential employees can use these scales to determine whether an 
individual’s skills match the skill requirements of a particular job, while educators can use them 
to determine how to best prepare students for the workplace. 

Assessments 

The Work Keys assessments enable employers and individuals to identify personal skill 
levels. Work Keys assessments are criterion-referenced, rather than norm-referenced. That is, an 
examinee’s performance on the assessments is compared to an established scale or standard (e.g., 
the proficiency level of a skill required for competent performance of a job), rather than against 
the test performance of others standardized along a normal distribution. Each assessment consists 
of four or five levels and each successive level is more complex than the previous level. 

Operational assessments have been developed for each of the following skill areas: Reading 
for Information, Applied Mathematics, Listening, Writing, Teamwork, Applied Technology, 
Locating Information (reading and interpreting graphically presented material such as tables, charts, 
graphs, etc.), and Observation. Several steps are taken to ensure that the Work Keys system 
assessments satisfy the needs of employers and educators and that the Work Keys assessments are 
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reliable and fair. First, ACT staff members with the help of representatives from business, 
industry, labor, and education develop test specifications. This process involves building the "blue 
print" for the assessment Before items are written, an overall structure is defined for what skill 
the assessment is going to measure and how the complexity of the items will increase as the skill 
levels increase. 

Once the general test blue print is developed, ACT staff members write sufficient numbers 
of prototype items to create one full length test form. This is administered to at least two groups 
of high school students and two groups of employees. Based on the information from the 
prototype administration, the test development specifications are adjusted and, if needed, 
prototyping is done again. 

Once the prototype is successful, a large number of items are written by item writers and 
edited by ACT staff members to meet content, cognitive, and format standards. These items are 
used to construct pretest forms of the assessments. During pretesting, approximately 2,000 
examinees respond to each item enabling an evaluation of each item's psychometric properties. 
The resulting information is used to create steps or "levels" in the assessment which are far enough 
apart to be statistically distinguishable, yet close enough together to provide useful information 
upon which employers, educators, and examinees can make decisions. Item statistics also give 
clues to possible problems with item content Items identified by the statistics as potentially 
problematic are reevaluated by ACT test specialists and, when appropriate, by qualified reviewers 
external to ACT. All items are also subjected to a differential item functioning (DIF) analysis 
(Dorans & Holland, 1993), a statistical analysis used to test for possible race and gender bias, prior 
to their inclusion in operational forms. In addition to the empirical data collected, all the items are 
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examined by qualified reviewers to ensure that they are not biased against or offensive to minority 
group members; other qualified reviewers examine the items for content accuracy. Finally, ACT 
staff members construct operational forms of the assessment using the reviewed items. 

Job Profiling 

Work Keys job profiling is a job analysis procedure which identifies the Work Keys skills 
and the levels of those skills needed to perform a particular job competently. The process consists 
of both a task analysis and a skills analysis. The resulting job profile can be used as the basis of 
the content validity of the Work Keys system, for identifying training needs, and for making hiring 
and promotion decisions. 

Each job profile is generated through a computer-assisted job analysis procedure. The 
software developed for this purpose includes a database of selected jobs and tasks from the 
Dictionary of Occupational Titles (U.S. Department of Labor, 1991). The jobs selected for this 
database have high levels of current employment and high growth potential for future employment 
To prepare for the task analysis phase of job profiling, the profiler uses the software to prepare an 
initial task list for the job being profiled. Then, during the job profiling meeting, a group made 
up of workers in the job (subject matter experts (SMEs)) are asked to edit the tasks until the 
revised list accurately depicts their job as it is performed in their company. After examining the 
modified task list carefully, the SMEs* rate the tasks according to importance (the significance of 
the task to overall job performance), and relative time spent (the amount of time spent performing 
this task compared to that spent on other tasks). Based on these ratings, the SMEs determine 
which tasks are most critical to their job. 
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This final task list is then used in the skills analysis phase of job profiling. For each skill, 
the SMEs are presented with the definition of that skill and are asked to identify the tasks requiring 
some level of that skill. Following a brief discussion among the SMEs, the profiler distributes a 
detailed description of one level of the skill to the SMEs. This description includes a definition 
of the skill level, and two examples of problems or situations at that level. The level definition 
is derived from the skill scale and is consistent with the test specifications for that skill’s 
assessment, and the examples presented are either previously pretested assessment items or are 
written to meet the test specifications for that level. Using a prepared rating form, the SMEs then 
indicate whether their job requires skill of greater than, less than, or about the same level of 
complexity as the examples. This process of reviewing levels continues until the SMEs come to 
a consensus regarding the level of each skill required for the job as a whole. The final product of 
this process is a document listing the most important tasks an individual in the job must perform 
and, for each relevant skill area, the skill level required for the job. Because job profiling uses the 
same skill scales as the assessments, the skills analysis conducted by the SMEs also establishes the 
appropriate "passing" score (i.e., level of proficiency required on the assessments). 

Instructional Support 

The Work Keys instructional support component provides trainers, curriculum developers, 
and others involved in the job-training process with materials that will facilitate their efforts to help 
learners improve their workplace skills. A series of instructional guides called the Targets for 
Instruction, designed to aid the development of appropriate curricula and effective instructional 
strategies for teaching the Work Keys skill areas, are central to this component The Target for 
Instruction developed for each skill area is intended to get the educator/instructor started in 
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developing instructional materials and services for generic employability skills. The Targets do 
not tell instructors how to teach, but instead outline the skills and the levels of those skills that are 
assessed by Work Keys so instructors know what skills have been identified as important by 
employers. 

Reporting 

The Work Keys system reporting component facilitates the distribution of information to 
individuals, educators, and employers. This information can help individuals make career choices, 
educators evaluate curricula and provide students with career guidance, employers plan training 
programs and screen prospective employees, and policy makers form decisions. 

Introduction to the Oklahoma Department of Human Services Validation Study 

The Oklahoma Department of Human Services (DHS) has expressed an interest in 
expanding its human resource functions within the social worker job family and is participating in 
this empirical validation study to determine the degree to which Work Keys system services will 
address its needs. Specifically, DHS is interested in exploring the use of the Work Keys system 
for training purposes and perhaps for other purposes such as selection at some point in the future. 

As the potential of the Work Keys System as a training tool was a primary interest, plans 
were made to test a sample of currently employed social workers so that their assessment scores 
could be compared to the skill profiles for the three jobs. As this would par tiall y complete the 
process for conducting a concurrent validation study, the decision was made to collect performance 
data so that the empirical relationship between social workers’ scores on the assessments and 
supervisory ratings of job performance could be studied. There are several different ways of 
establishing validity and the Work Keys system was developed with content validity in mind. 
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meaning that the content of the assessment should link to the content of the job. The criterion- 
referenced nature of the Work Keys skill areas and their assessments aids in establishing this 
linkage through the job profiling process. While this information alone is sufficient to establish 
content-related validity, DHS and ACT have gone a step further with this project to conduct an 
empirical research study. The term concurrent is used to describe this study because current 
employees are serving as the population of interest. 

A decision made early in the project involved whether to include all of the operational 
Work Keys skills. While all parties wanted as much information as possible about basic skills 
required to be a social worker, time constraints were also an issue. After some debate, the 
following skills were included: Reading for Information, Applied Mathematics, Teamwork, 

Locating Information, Listening, and Writing. The Applied Technology skill was omitted due to 
the low probability that it would be relevant to the social worker jobs. Observation was not 
included because the materials related to this skill were not yet available when this project started. 

The project consisted of (1) conducting job profiling for each of the social worker 
classifications to establish content validity and skill standards, (2) administering assessments to a 
sample of social workers from each classification, and (3) collecting job performance data for the 
social workers who took the assessments so that the relationship between performance on the 
assessments and performance on the job could be studied. Each step is discussed further below. 

Procedure 

Job Profiling 

Three classifications of social workers participated in the job profiling component of this 
project: Social Worker I (SWI), Social Worker II (SWII), and Social Worker HI (SWIII). The 
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SWIs and SWIIs perform many of the same tasks and form the backbone of the social worker 
hierarchy. These jobs differ primarily by time in grade and by the size of the caseload carried. 
At the time profiling was conducted, one hundred twenty-six individuals were employed as SWIs 
and 1,445 individuals were employed as SWIIs. The SWlHs form a significantly smaller group 
(n=43). They are responsible for removing adults from situations of abuse and neglect 

Due to the number of individuals employed as social workers in the state, two separate and 
independent job profiling sessions were held for each classification. Even though the SWin group 
was quite small, two separate sessions were held to keep the manner in which this project was 
implemented consistent across the classifications. An effort was made to recruit SMEs who were 
representative of the gender and racial makeup of the job incumbents. 

The six job profiling sessions for this project were conducted during a two-week period in 
January, 1995. The authorized job profilers who conducted the sessions found that the majority 
of participants took the profiling experience seriously and appreciated the opportunity to talk about 
their jobs. In addition, a high degree of agreement was found within and across five of the six 
groups. The performance of the sixth, "outlier," group is discussed below. 

Table 1 shows the job profile generated by each group of SMEs for each classification. The 
most obvious discrepancy is between Groups 1 and 2 for the SWI job. In this case. Group 2 
appears to be the outlier group. The first round of profiling (Le., the profiling done by Group 1 
for each classification) and discussions with individuals in DHS indicated that the SWI and SWII 
jobs are very similar. In fact, the profiles generated by the first groups of SMEs for the two jobs 
differed only in the level of Teamwork skill required. The profile generated by the second group 
of SWEs is also very similar to the ones developed by the first groups of SWIs and SWIIs. The 
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discrepancies between the two SWI groups may be due to the atypical composition of Group 2: 
only five individuals were able to participate in this session for the SWI job. The small size of 
this group made it easier for a particular participant to dominate the discussion. This individual 
consistently urged the group to select a higher skill level for each skill than the group might have 
selected otherwise. Because of this mitigating factor, it was decided that information from Group 
2 concerning the SWI job would not be included in the final analysis, allowing the profile 
determined by Group 1 to stand. 

Because the profiles from the two groups of SWIIIs did not differ, only one discrepancy 
remained. The profiles generated by the two groups of SWUs differed in the level of the Reading 
for Information skill required. The usual procedure to resolve such a skill level discrepancy is to 
convene representatives from each group for another meeting to reconcile the difference. However, 
because the social workers were a sample from around the state, it was not feasible to physically 
reconvene them for a second meeting. Therefore, the meeting was held via conference call. 

Prior to the conference call, participants were mailed packets containing a cover letter 
explaining the purpose of the conference call, along with the relevant skill definition and level 
definitions. During the conference call, participants were asked to reconcile the difference between 
the levels of the Reading for Information skill that were determined to be necessary by the two 
groups of SWHs during the job profiling sessions. This involved referring to the task fists 
generated by the groups, discussing the nature of the material read on the job, and determining the 
amount of support social workers receive when required to read complex m a terials such as legal 
documents. The participants decided that Level 6 was the appropriate standard for this skill as 
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support is available to assist SWIIs with complex material such as legal documents. The final 
profiles for each Social Worker classification are shown in Table 2. 

Table 2 shows the pattern of complex generic skills required by social workers. This is 
consistent with the fact that Social Worker is a professional occupation needing a high level of 
education, and that Social Workers are often faced with demanding circumstances. The three 
classifications of Social Workers are quite similar in the skill levels needed. As mentioned earlier, 
SWI and SWII are separated only by the level of the Teamwork skill needed, where SWO is one 
level higher than SWI. Somewhat more variation is found in the skill levels for SWIHs. They 
require one level higher in the Reading for Information and Teamwork skills than do SWIIs. The 
higher reading skill level may be due to their frequent dealings with, and even writing of, legal 
documents. Several S Wills commented that they must frequently act as attorneys when they 
appear in court The SWIHs also commented that because many of their actions are dictated by 
state laws, they are frequently put into situations where they are the change agents and their 
relationships with others become strained (e.g., clients and caregivers), which requires them to have 
a high level of Teamwork skill. 

Assessment Administration 

The second phase of this project involved administering the assessments for the six slrill 
areas used in this project to a sample of currently employed social workers within the state. As 
an emphasis was placed on making sure that the sample of the social workers participating in the 
testing was representative of social workers employed by the state, this phase of the project 
required indepth planning and the assessments were administered in November, 1995. In order to 
take all six assessments, each individual participating in the testing was asked to attend two test 
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sessions. The Reading for Information and Teamwork assessments were administered in the first 
session and the remaining assessments were administered in the second session which was held a 
week later. A total of 300 social workers were invited to participate in the assessment 
administration. Test data indicate that 272 social workers attended both test sessions while 11 
social workers attended only the first test session. The remaining 17 individuals apparently were 
not able to attend either of the testing sessions. 

Table 3 provides a summary of the characteristics of each assessment (including reliability). 
The assessments for the Reading for Information, Applied Mathematics, and Locating Information 
skill areas are in a paper-and-pencil, multiple-choice format The Teamwork assessment is 
administered using a videotape. Examinees are presented with a series of videotaped scenarios 
showing work teams in various situations. Following each scenario is a set of multiple-choice 
questions to which examinees respond on answer sheets. 

The Listening and Writing assessment is in a constructed-response format Specifically, 
examinees are asked to listen to a series of six audiotaped messages and then write messages or 
summaries based on the information heard. This simulates the taking of messages or other 
information that must be written down so that it can be given to a third party or referred to at a 
later point in time. The examinees’ written responses are scored by one set of scorers for the 
accuracy of the information recorded (the Listening score) and they are scored by another set of 
scorers for spelling, punctuation, and writing style (the Writing score). Appendix A provides the 
skill definitions and level definitions for the skill areas included in this project 
Collection of Job Performance Data 

The next phase of the project was to distribute performance rating forms to the immediate 
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supervisors of the social workers who participated in the testing phase of the project These 
performance rating forms were constructed using the task lists determined during the job profiling 
portion of this project Specifically, forms were created which asked supervisors to rate their social 
worker’s performance on each task, on a nine-point scale, and then to rate overall job performance 
using the same scale. Due to the similarity between the tasks performed by SWIs and SWIIs, the 
same performance appraisal form was used for both jobs. As the SWiLl job involved a number of 
tasks not cited by the SWIs and SWIIs, a separate form was generated for this job. 

The performance rating forms were mailed to supervisors in January, 1996. Supervisors 
were given several weeks to complete the forms and the final completed forms were received by 
the end of March, 1996. Rating forms were received for 189 (32 SWIs, 150 SWIIs, and 7 SWIIIs) 
social workers. In some cases (n=24), rating forms were received for social workers who had not 
taken the assessments. The elimination of these forms resulted in usable forms for 165 social 
workers (25 SWIs, 135 SWIIs, and 5 SWIIIs). As forms had been mailed to supervisors of the 283 
individuals who took the assessments, this represented a return rate of fifty-eight percent 

The small number of completed SWill forms may be due to a clerical error that occurred 
when the forms were mailed to the supervisors. In several cases, supervisors of SWIIIs were 
mailed the performance appraisal form for SWIs and SWIIs rather than the SWIH form. This 
resulted in data that could not be used. 

Results 

Only employees who took the assessments and who were also evaluated by their supervisors 
were included in the study sample (58% of the total assessment sample). The elimination of cases 
with missing data resulted in data for 20 SWIs, 120 SWIIs, and 3 SWIIIs (50.5% of the total 
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assessment sample) being retained for analysis purposes. An examination of the distributions of 
the study sample and of the total sample showed that there were no differences, indicating no 
response bias. 

For the study sample, the majority of social workers in all three classifications were female 
(SWI, 80%; SWH, 64.7%; and SWin, 100%). The primary ethnic group across all three 
classifications was Caucasian (SWI, 55%; SWII, 80.7%; and SWin, 100%). The majority of the 
remaining social workers reported African-American or American Indian/Alaskan Native as their 
ethnic group (SWI: African-American, 20%; American Indian/Alaskan Native, 15%; and SWH: 
American Indian/Alaskan Native, 7.6%; African-American, 5.9%). Many social workers reported 
being college graduates (SWI, 45%; SWH, 47.9%; and SWin, 33.3%) while some had post- 
graduate training (SWI, 30%; SWH, 34.5%; and SWIH - 33.3%) 

Analyses were conducted to determine (1) the distribution of scores for each social worker 
classification for each assessment taken; (2) the number of individuals in each classification who 
met or exceeded the profile for their classification; and (3) the correlation coefficients between test 
performance and performance on the job. Each set of analyses will be discussed separately below. 
Distribution of Scores 

Tables 4A-4C provide the distribution of the level scores on each assessment for the study 
sample. The skill levels set by social workers participating in the job profiling portion of this 
project were used as the passing score or the criterion for the various assessments. 

The profiled level for the Reading for Information skill area for SWIs and SWUs was 
determined to be Level 6. The data indicate that 60% of the SWIs and 72.5% of the SWIIs scored 
at or above this leveL Thus the majority of the individuals in these job classifications met or 



Validation 16 



exceeded the skill level set for their jobs. This was not the case, however, for the SWIIIs. While 
the Reading level was set at Level 7 for the SWIIIs, all individuals in the small sample retained 
for data analyses scored at Level 6. Due to the small sample, it is not possible to draw conclusions 
about the larger SWm population. 

Similarly, the majority of social workers scored at or above the profiled level for the 
Applied Mathematics skill. Specifically, all three jobs profiled as requiring Level 4 of this skill, 
and 80% of SWIs, 90% of SWIIs, and 100% of SWIIIs scored at or above that level. 

The social workers’ scores on the Teamwork assessment also showed that the majority of 
SWIs and SWIIs tested as at or above the profiled levels for the jobs. The SWI job profiled as 
requiring Level 4 of this skill, and 95% of the tested SWIs scored at or above this level. The SWII 
job was set as requiring Level 5 of the Teamwork skill and 54.2% of the examinees scored at or 
above Level 5. The SWm job profiled at the highest level of this skill (Level 6) and none of the 
three social workers comprising the SWin job scored at that level. Again, due to the small sample 
size for SWIIIs, it is difficult to know the status of the larger population on the Teamwork skill . 

The Locating Information skill assessment scores showed some deficiencies in social worker 
skills. The social workers participating in the job profiling process determined that all three jobs 
required Level 5 of this skill (reading charts, graphs, tables, etc.). The test results showed that 35% 
of the SWIs, 36.7% of the SWIIs and 33.3% of the SWIIIs (one individual) scored at Level 5. A 
larger percentage of examinees, however, scored at Level 4 of this skill (SWI, 45%; SWII, 54.2%; 
and SWm, 66.7%). 

During the job profiling meetings, social workers indicated that they were using material 
similar to the Level 5 examples shown to determine eligibility for various programs. <3iven that 
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the social workers were consistent in their decision that Level 5 of this skill is required for the jobs 
and the linkage of the skill to specific tasks of the job, it seems quite possible that a skill gap has 
been identified. In this case, training in this skill area will better equip social workers to work with 
the graphical material they encounter in their job. While it is clear that some tasks of the job 
require Level 5 of this skill, it was also clear that some tasks of the job do not require reading 
graphical material. It could be that the portion of the job related to Level 5 was over-emphasized 
during the profiling process. However, as the same decision was reached by independent groups, 
this seems an unlikely explanation. 

Results also showed that the social workers participating in the testing found the Listening 
and Writing assessment to be challenging. While all three social worker jobs profiled as requiring 
Level 5 of the Listening portion of the Listening and Writing assessment, the highest level of this 
skill, none of the examinees scored at this level. In fact, only 20% of the SWIs, 8.3% of the 
SWHs, and 33.3% (one individual) of the SWIIIs scored at Level 4. Examining the results for the 
next level indicates that the majority of examinees mastered Level 3 of the Listening skill (SWIs, 
70%; SWHs, 84.2%; SWm, 66.7%). 

These results may be due to the relationship between the manner in which the Listening and 
Writing assessment is administered and the type of listening done by social workers on the job. 
The Listening and Writing assessment is administered by having examinees listen to an audiotape 
which presents six prompts which simulate the taking of phone messages or other information that 
must be written down so that it can be passed along to a third party or referred to at a later point 
in time. For Listening, the recorded information is then scored for the accuracy of the material that 
was recorded. At Level 5, all the information, including all details are recorded. The type of 
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listening most often mentioned by the social workers during the job profiling meetings was that 
involved in the interviewing of clients and individuals associated with clients. The social workers 
noted that this information must be recorded so that it can later be included in reports and used to 
complete various forms. It may be that there are qualitative differences between the type of 
listening demonstrated on the assessment and the type of listening done on the job. Specifically, 
social workers may record only the information conveyed by clients and others which is needed 
to write reports and complete forms. They may not be required to record all the information 
conveyed by others to perform their jobs competently. 

The Writing portion of the Listening and Writing assessment showed results similar to those 
for the Listening portion. The Listening and Writing assessment is administered as one assessment 
from which two scores can be derived. The examinees’ written responses are scored by two 
independent scorers for the accuracy of the information recorded and then the responses are scored 
by two different independent scorers for writing mechanics (e.g., spelling, punctuation, etc.). While 
all three jobs profiled as requiring Level 4 of this skill, results indicate that 20% of the SWIs, 
32.5% of the SWIIs, and 33.3% of the SWIIIs (one individual) scored at this level. As with 
Listening, the majority of examinees scored at Level 3 (60% of SWIs, 58.3% of SWIIs, and 66.7% 
of the SWIIIs). 

As the social workers were all consistent in their decision that Level 4 of the Writing skill 
is required for the social worker jobs, it seems likely that a skill gap has been identified. As the 
social workers indicated that they are responsible for writing reports and completing paperwork, 
a writing training program aimed at these specific aspects of the job may be warranted. 
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It is also possible that the discrepancy between the profiled level of Writing and the 
assessment results reflect intraoffice differences in what is considered acceptable writing. During 
the job profiling meetings, the social workers indicated there seemed to be differences across the 
various offices as to what was acceptable writing. Another explanation may be that examinees did 
not attend closely to the assessment directions which indicate that they should pay attention to the 
manner in which they wrote their responses. As the assessment simulates having to listen to 
spoken information and then record it, the social workers may have written as if they were taking 
messages where writing mechanics are often not of primary importance. 

Percentage Meeting or Exceeding Profile 

The data were also analyzed to determine the percentage of social workers who met the 
profiles for their jobs. As six skill areas are included in the profile for each job, it was expected 
that it would be extremely difficult for any individual to meet the specified level for all the skills 
for their job. It is to be expected that when so many skill areas are being examined, some 
individual weaknesses will become apparent 

As expected, no single individual met the profiled levels for all the skills. As the above 
discussion highlighted, the Listening and Writing assessment was the most challenging assessment 
for these social workers and the differences between the levels set by social workers during job 
profiling and the assessment results was the primary cause for no one meeting the entire profile. 
Correlation Coefficients: Validity 

Tables 4A-4C present the distribution of the assessment scores for the three social worker 
classifications and Table 5 presents the distribution of the performance ratings for the three 
classifications. The final performance rating made by supervisors, that of overall job performance. 
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was used as the criterion. The data in Table 5 indicate that it was at the midpoint of the scale 
("5") that the frequency ratings increased from the lower ratings. Using the midpoint as the job 
performance cutoff, one finds that 85% of the SWIs, 91.7% of the SWIIs, and 100% of the SWms 
were evaluated by their supervisors as having job performance that is "good," "very good," or 
"understanding." The performance rating form was further evaluated by calculating coefficient 
alpha to provide an estimate of the reliability of the form. The analysis produced a Coefficient 
Alpha value of .93. 

For the purposes of the correlational analyses, the data from the SWIs and the SWIIs were 
combined. As the introduction to this project indicated, the two jobs are very similar. The 
similarity between the two jobs was also reflected by the profiles being identical except that the 
SWIIs indicated they required a higher level of the Teamwork skill. The SWms were not included 
in the analyses as their job is considered qualitatively different from SWIs and SWIIs. 

The correlational analyses indicated that there was a statistically significant correlation 
(using a one-tailed test of significance) between assessment scores and supervisory ratings of job 
performance for the Reading for Information. Applied Mathematics. Locating Information, and 
Listening and Writing skill assessments. The correlation for the Teamwork assessment was not 
statistically significant Table 6 provides the specific correlations and shows that the statistically 
significant correlations ranged from .19 to .23 (p < .05). One of the criticisms of concurrent 
validation studies has long been that the use of current employees as the relevant population results 
in test scores and job performance that is far less variable than if a random sample of job 
applicants was used. The reasoning being that individuals already on the job were hired because 
they were expected to succeed on the job so they are more likely to score well on tests related to 
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the job and more likely to get high ratings of job performance than would individuals not hired. 
The end result is a dataset which is probably less variable then that collected from a random 
sample. This type of score distribution constitutes what is commonly called "restriction of range" 
which acts to decrease the correlation coefficient. 

As random samples are not often available when conducting applied research, a formula is 
available for estimating what the correlation coefficient would have been had a more variable 
sample been available (Hunter, Schmidt, & Jackson, 1982). Applying this formula requires 
knowing the correlation between the predictor and the criterion, the standard deviation of the 
predictor in the study sample, and in the population (Reading for Information : study, .98; 
population, 1.16; Applied Mathematics: study, 1.03; population, 1.21; Teamwork : study, .76; 
population, 1.02; Locating Information : study, .69; population, .91; Listening : study, .42; 
population, .69; Writing : study, .61; population, .86. Standard deviations from a large scale 
administration of the Work Keys assessments (n = 84,092) were used as estimates of the population 
standard deviations. The application of this formula resulted in the following estimates for the 
validity coefficients in the population: Reading for Information. .19; Applied Mathematics. .27; 
Locating Information. .25; Listening. .30; and Writing. .27. Corrections for restriction of range 
were calculated only for the statistically significant uncorrected correlations, therefore Teamwork 
was not included. 

The explanation for the lack of a statistically significant correlation between the Teamwork 
assessment and job performance may be related how teamwork is defined. It may be that 
Teamwork, as defined by Work Keys, differs from the teamwork performed by social workers. 
ACT’s definition of teamwork involves an employee’s skill in choosing behaviors and/or actions 
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that simultaneously support the relationship among team members and lead toward the 
accomplishment of work tasks. While social workers work with others to accomplish some of the 
tasks of their job, they may also spend enough time working individually that a test of teamwork 
skill will not be predictive of job performance. 

Summary and Recommendations 

The outcomes of this concurrent validation study provide DHS with quite a bit of 
information, and, as a result, give the department several options. The finding of statistically 
significant correlations between scores on the Reading for Information. Applied Mathematics. 
Locating Information, and Listening and Writing assessments and the supervisory ratings of overall 
job performance indicate that DHS should allocate their resources toward these skill areas. While 
Teamwork is a valued skill in many employment settings, the results of this study suggest that the 
manner in which ACT has defined Teamwork is not related to job performance as a social worker. 

As the use of Work Keys system services for training purposes is a primary concern for 
DHS, the implications of these data for training will be discussed first Specifically, the data 
indicate that skill gaps may exist for currently employed social workers in the Locating 
Information, Listening, and Writing skill areas. Although providing training in all five skill areas 
is the ideal course of action, many organizations today are experiencing resource limitations. If 
DHS is facing similar challenges, the Locating Information, Listening, and Writing skills should 
receive a higher priority for training purposes than the Reading for Information or Applied 
Mathematics skills . 

An additional issue is training for new hires. Given that current social workers who were 
rated by their supervisors as being good to outstanding at their jobs had problems with the Locating 
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Information, and Listening and Writing assessments, it seems quite possible that new hires will find 
these skill areas to be problematic as well. While incumbent social workers did well on the 
Reading for Information and Applied Mathematics skill assessments, this does not necessarily mean 
that new hires will do as well. For diagnostic purposes, DHS may want to administer the 
assessments to all new hires so areas of individual strengths and weakness can be identified and 
individual development plans can be implemented. 

The data reported here also have implications for selection and promotion. Whenever 
personnel selection issues are discussed, it is necessary to consider the federal government’s interest 
in employee selection procedures. In a nutshell, the various guidelines (e.g., Uniform Guidelines 
on Employee Selection Procedures. 1978: Principles for Validation and Use of Personnel Selection 
Procedures. 1978) indicate the need for an organization to investigate the validity of its selection 
procedures and to document the results of such investigations. The Work Keys system was 
developed with content validity in mind, meaning that the content of the assessment should be 
linked to the content of the job. For the Work Keys system, this linkage is achieved with the job 
profiling process. While this information alone is sufficient to establish content-related validity, 
DHS and ACT have gone a step further with this project to investigate the empirical relationship 
between assessment scores and job performance for research purposes. 

Regarding promotion, one option is to use the skill information in a career development 
model. In this scenario, the assessments can be administered to social workers to identify areas 
where individuals have skill gaps. This information could be used to alert individuals to areas in 
which they need to improve to perform their current jobs. With the job profiles for the higher level 
jobs, individuals would know what skill levels they need to achieve to increase their likelihood of 
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being promoted. If resources permit, DHS may want to provide training in these generic skills to 
employees to give everyone an equal opportunity to improve their skills. 

Using Work Keys assessment data for selection purposes would involve comparing a job 
applicant’s test scores to the required levels of the five skills. The relevant issue is what levels to 
use as the cutoffs. The discussion regarding the score distributions highlighted that using the 
profile levels set by the social workers for the Locating Information, Listening, and Writing skills 
would eliminate more than half of the individuals in the study sample. It also suggests that it may 
be difficult to find a sufficient number of applicants who meet the profiled levels. If this is the 
case, the suggested course of action to increase the applicant pool is to drop the required levels for 
the Writing and Locating Information skills by one level, to levels that were achieved by the 
majority of social workers. The next course of action would be to provide training to the new hires 
to help them improve their skills to the levels required for their new jobs. 

The Listening skill raises additional issues. As there is the possibility that Work Keys’ 
Listening skill may be slightly different from the type of listening done on the job, the most 
conservative course of action would be to revisit the Listening skill with groups of social workers 
to reexamine its linkage to the tasks of the job. If DHS prefers not to revisit the profiling and 
decides to use the Listening skill for hiring purposes, the department will need to follow the same 
course of action outlined above for the Locating Information and Writing skills. In the case of the 
Listening skill, it might be necessary to lower the Listening level by two levels to increase the size 
of the applicant pool. A reexamination of the data reported in this study indicates that dropping 
the required Locating Information and Writing skill levels by one level and dropping the Listening 
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level by two levels results in 50% of the SWIs and 42.5% of the SWIIs meeting the profiles for 
their jobs. 

In summary, this study has examined the usefulness of Work Keys system services for 
addressing human resource management needs for social workers in DHS. Results indicate that 
each of the skills, with the exception of Teamwork, could be used to address training issues, which 
is DHS’s primary concern. This study also provides data to assist DHS with deciding how to use 
limited resources. As testing on five skill areas can be a rather time consuming and costly process, 
the results of the analyses indicate that DHS may want to consider training for current employees 
in the areas of the Locating Information, Listening, and Writing skills. It is conceivable that new 
hires may have weaknesses in additional areas which suggests that administering all the 
assessments will provide diagnostic information. The results of the correlational analyses also 
provide additional support for using the five skills in a selection model. 
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Table 1 

Results of the Job Profiling Sessions for Each Social Worker Classification 



Social Worker I 





Number 

of 

SMEs 


Reading for 
Information 
(3-7) 


Locating 

Information 

(3-6) 


Teamwork 

(3-6) 


Listening 

G-5) 


Writing 

G-5) 


Applied 

Mathematics 

(3-7) 


Group 1 


9 


6 


5 


4 


5 


4 


4 


Group 2 


5 


7 


6 


6 


5 


5 


5 



Social Worker II 





Number 

of 

SMEs 


Reading for 
Information 
(3-7) 


Locating 

Information 

(3-6) 


Teamwork 

(3-6) 


Listening 

G-5) 


Writing 

G-5) 


Applied 

Mathematics 

(3-7) 


Group 1 


6 


6 


5 


5 


5 


4 


4 


Group 2 


7 


7 


5 


5 


5 


4 


4 



Social Worker HI 





Number 

of 

SMEs 


Reading for 
Information 
(3-7) 


Locating 

Information 

(3-6) 


Teamwork 

(3-6) 


Listening 

G-5) 


Writing 

(1-5) 


Applied 

Mathematics 

(3-7) 


Group 1 


8 


7 


5 


6 


5 


4 


4 


Group 2 


6 


7 


5 


6 


5 


4 


4 



Note. The numbers in parentheses under each skill name reflect the range of skill levels for that particular 
skill. 
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Table 2 

Final Job Profiles for Each Social Worker Classification 





Reading for 
Information 
(3-7) 


Locating 

Information 

(3-6) 


Teamwork 

(3-6) 


Listening 

(1-5) 


Writing 

d-5) 


Applied 

Mathematics 

(3-7) 


Social 
Worker I 


6 


5 


4 


5 


4 


4 


Social 
Worker II 


6 


5 


5 


5 


4 


4 


Social 

Worker 

m 


7 


5 


6 


5 


4 


4 



Note. The numbers in parentheses under each skill name reflect the range of skill levels for that 
particular skill. 
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TABLE 3 

Assessment Characteristics Validation 
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TABLE 4A 

Performance of Social Worker Is on the Work Keys Assessments Validation 
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TABLE 4B 

Performance of Social Worker IIs on the Work Keys Assessments Validation 
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TABLE 4C 

Performance of Social Worker IDs on the Work Keys Assessments Validation 
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TABLE 5 

Overall Job Performance Ratings of Social Workers Validation 
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TABLE 6 

Correlation Coefficients (Work Keys assessments correlated with 
supervisory ratings of overall job performance) 
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Appendix A 
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Reading for Information Skill 



The Reading for Information skill is an employee’s skill in reading and understanding work— related 
reading materials. In evaluating the level of the Reading for Information skill required for the tasks of 
the job, consider 

• the difficulty of the materials employees must read (e.g., straightforward announcements using 
simple vocabulary or complex legal documents which describe complicated procedures and 
include technical language or specialized language), and 

• how hard it is for employees to find and make use of the information they need in the reading 
materials (e.g., employees are required simply to use information stated directly or they must 
generalize and draw conclusions from the information). 

Keep in mind that this skill does not include the skill in reading charts, graphs, tables, forms, blueprints, 
maps, or instrument gauges. 

Five levels of difficulty are described, ranging from Level 3, the least complex, to Level 7, the most 
complex. Examples of reading materials associated with each level of the skill are provided. Higher 
levels of difficulty include the skills described in lower levels. 



Employees must read basic company policies, procedures, and announcements. These workplace reading 
materials are short, simple, and use elementary vocabulary. All information employees need in order 
to choose an appropriate course of action is stated clearly in the materials; employees do not need to 
read between the lines. 

Employees are required to 

• understand the meaning of words that are defined in these workplace reading materials. 

• figure out the meaning of elementary words that are not defined in these reading materials. 

• understand the main ideas and straightforward details from these reading materials. 

• understand when to perform each step in a series from reading directions. 

• be able to apply instructions outlined in these reading materials to situations described in these 
reading materials. 



Level 3 
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Level 4 



Employees must read straightforward company policies, procedures, and announcements which contain 
a number of details and describe procedures which involve several steps. Many of the reading materials 
describe policies and procedures which require employees to take changing circumstances into account 
in identifying the course of action which will best accomplish their goals. 

Employees are required to 

• notice important details in these reading materials. 

• figure out the meaning of words that are not defined in these reading materials. 

• apply instructions, some of which involve several steps, to situations described in these reading 
materials. 

• take changing circumstances into account in order to decide what to do. 



Level 5 



Employees must read moderately detailed and complicated company policies, procedures, and 
announcements. These reading materials contain words and phrases that may be specialized (jargon and 
technical language) or words that have several meanings. All of the information employees need is 
stated clearly in the reading materials, but the employees must consider several factors in order to 
identify the course of action that will accomplish their goals. 

Employees are required to 

• understand the paraphrased definition of specialized words or phrases (jargon or technical terms) 
defined in these reading materials. 

• use jargon or technical terms appropriately in describing situations stated in these reading 
materials. 

• understand the meaning of acronyms defined in these reading materials (an acronym is a word 
or collection of letters which stands for a longer phrase, such as HMO to mean Health 
Maintenance Organization). 

• figure out which definition of a word with multiple meanings is appropriate in the context of 
these reading materials. 

• apply information given in these reading materials to situations that are not directly described, 
but similar. 

• apply instructions or procedures with a number of steps to described situations. These 
instructions may include conditionals (if X happens, then you should do Y). 
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Level 6 

Employees must read difficult company policies, procedures, and announcements. These reading 
materials present complicated information; for example, they may include exceipts from regulatory and 
legal documents. These reading materials use advanced vocabulary, jargon, and technical terms to 
describe elaborate procedures and concepts. Most of the information employees need in order to identify 
an appropriate course of action is not clearly stated in the reading material. Thus, employees may need 
to determine the principles underlying the described situation and apply those principles to new situations 
not depicted in die reading material. 

Employees are required to 

• understand specialized words or phrases (jargon or technical terms) when used in an unfamiliar 
context. 

• apply complicated information to new situations. 

• figure out from context the less common meaning of a word with multiple meanings. 

• figure out the general principles underlying situations described in these reading materials and 
apply those principles to related situations. 

• understand implied details. 

• figure out the reasoning behind a procedure, policy, or communication. 



Level 7 



Employees must redd materials which are very difficult* the information is detailed, the concepts are 
complicated, and the vocabulary is difficult The jargon and technical terms used are not defined in the 
reading materials. Employees must generalize beyond stated situations, understand implied details, and 
figure out the reasoning behind stated policies and procedures. 

Employees are required to 

• figure out the definitions of difficult uncommon jargon or technical terms from the context of 
the reading materials. 

• figure out the general principles underlying described situations and apply them to situations 
neither described in nor completely similar to those in the reading materials. 
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Applied Mathematics Skill 



The Applied Mathematics skill is an employee’s skill in applying mathematical reasoning and problem- 
solving techniques to work-related problems. In evaluating the level of the Applied Mathematics skill 
required for the tasks of the job, consider 

• the types of mathematical operations employees must perform (e.g., single-step or multiple-step 
mathematical operations, conversions either within or between systems of measurement); 

• how the information in the problems is presented to employees (i.e., the information is presented 
in the order in which it is needed or it must be reordered); and 

• whether all the information employees need to solve problems is provided (or if they must derive 
some necessary information). 

Keep in mind that employees are in a workplace where they have calculators and conversion tables to 
assist them. 

Five levels of difficulty are described ranging from Level 3, die least complex, to Level 7, the most 
complex. Examples of the kinds of work-related problems associated with each level of the skill are 
provided. Higher levels of difficulty include and build on the skills described in lower levels. 



Level 3 



Employees are required to 

• do one step of mathematical operation (Le., addition, subtraction, or multiplication) on positive 
or negative numbers, as well as division of only positive numbers (e.g., 20). 

• change a number from one form to another, using whole numbers (e.g., 10), fractions (e.g., Vi), 
decimals (e.g., .75), or percentages (e.g., 12%); for instance, employees may be required to 

convert 1 to its equivalent percentage. 

For example, at this level employees may be required to add the prices of several products or to make 
the correct change for a customer. 
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Level 4 

Employees arc required to 

• do one or two mathematical operations, such as addition, subtraction, or multiplication, on 
several positive or negative numbers (e.g., 10, -2), as well as division of only positive numbers. 

• figure out averages (e.g., p°*“* 12 I), simple ratios (e.g., 1), simple proportions (e.g., cases), 
or rates (e.g., 10 mph) using whole numbers and decimals. 

• add commonly known fractions, decimals, or percentages (e.g., Vi, .75, 25%), or add three 
fractions that share a common denominator (e.g., 1 + i. + 1). 

• reorder verbal information before performing calculations. 

• read a simple diagram or graph to get the information needed to solve a problem. 

For example, employees may be required to calculate sales tax or a sales commission, or to figure rates 

of use or business flow. 



Level 5 



Employees are required to 

• look up a formula and change from one unit to another unit of measurement within a system of 
measurement (e.g., from ounces to pounds) or between systems of measurements (e.g., from 
centimeters to inches). 

• calculate using mixed units (e.g., 3.50 hours and 4 hours 30 minutes). 

• do several steps of logic and calculations, including division of negative numbers. 

• decide what information, c alculati ons, or unit conversions are needed to find a solution. 

• determine the best deal 

For example, employees may be required to calculate perimeters and areas of basic shapes (e.g., 
rectangles and circles), to o a i rnlatp. percent discounts or markups, to compare costs to determine which 
is the best deal, or to complete a balance sheet or order form that requires several math operations (e.g., 
total an order, and then calculate tax and shipping costs). 
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Level 6 



Employees are required to 

• set up problems and do several steps of calculations or conversions. 

• calculate using negative numbers, fractions, ratios, percentages, or mixed numbers (e.g., 12Vfe). 

• transpose a formula before calculating (e.g., 8X = 20 => X = i£.). 

• look up and use two formulas to change from one unit to another unit within the same system 
of measurement (e.g., 1 cup = 8 fl oz, 1 quart = 4 cups). 

• find mistakes in calculations, such as those required in lower levels. 

• determine the best deal and perform a further calculation with the result 

For example, employees may be required to calculate multiple rates, to find areas of rectangles and 
volumes of rectangular solids, or to solve problems that compare production rates and pricing schemes. 



Level 7 

Employees are required to 

• do several steps of reasoning and calculations. 

• solve problems involving more than one unknown, and nonlinear functions (e.g., rate of change). 

• find mistakes in multiple-step calculations. 

• figure out the information needed to solve a problem when the information presented is 
incomplete or implicit 

For example, employees may be required to convert between systems of measurement that involve 
fractions, mixed numbers, decimals, or percentages; to calculate multiple areas and volumes of spheres, 
cylinders, or cones; or to set up and manipulate complex ratios or proportions. 
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Teamwork Skill 



The Teamwork skill is an employee’s skill in choosing behaviors and/or actions that simultaneously 
support the relationships among team members and lead toward the accomplishment of work tasks. 
Employees must 

• recognize the goals of the team and 

• identify ways to accomplish them in increasingly complex situations, such as those where the 
resources needed to accomplish the task are not readily available. 

Four levels of difficulty are described, ranging from Level 3, the least complex, to Level 6, the most 
complex. Examples of the kinds of work-related teamwork problems associated with each level 
of the skill are provided on the attached pages. These examples are in the form of video scenarios 
based on the actual demands of the workplace. The scenarios present teams in various workplace 
settings with various problems or requirements. Higher levels of difficulty include the skills described 
in lower levels. 



Level 3 

Employees are required to recognize the behaviors or actions which would best support the team and 
contribute to work performance when faced with simple work situations involving one problem or one 
issue that needs to be handled. In these work situations, the team goals and consequences are clear, all 
the resources needed to deal with the problems are available, and the relationship among team members 
is good. 

Employees may be required to 

• understand the goal that the team is trying to accomplish and how to work with other team 
members to accomplish that goaL 

• choose actions that support the ideas of other team members and try to use their suggestions to 
accomplish team goals. 

• determine if the team is having problems finishing a task and figure out what is causing these 
problems. 



Level 4 



Employees are required to recognize the behaviors or actions which would best support the team and 
contribute to work performance when faced with work situations involving several problems or issues 
to be handled. In these work situations, the goals and consequences are not totally clear, some of the 
resources needed to deal with the problems are not available, and/or the team members have competing 
concerns or needs (there is underlying tension, but no confrontation). 
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Employees may be required to 

• organize tasks and schedule time in a way that will help accomplish team goals efficiently and 
effectively. 

• choose a course of action that indicates consideration of what is said by team members. 

• identify behaviors that show appreciation for the personal and professional qualities of other 
team members and respect for their differences. 



Level 5 



Employees are required to recognize the behaviors or actions which would best support the team and 
contribute to work performance when faced with new work concerns involving many subtle and 
competing problems and issues to be handled. In these work situations, the team goals and 
consequences are unclear, many of the resources needed to deal with the problems are not available, and 
the team members have competing concerns and needs (tension is clearly present among the team 
members). 

Employees may be required to 

• identify courses of action that distribute the workload to the team members effectively. In 
taking on tasks and sharing tasks with others, employees must consider how best to use team 
talents to accomplish team goals. 

• choose approaches that encourage and support the efforts of other team members to further team 
relationships and/or task accomplishment 

• consider the possible effects of alternative behaviors on both team relationships and team 
accomplishments and select the one which will best help the team meet its goals (each team 
member’s behavior affects the other team members more at this level). 



Level 6 



Employees are required to recognize the behaviors or actions which would best support the team and 
contribute to work performance when faced with work situations involving complex problems and issues 
to be handled. In these work situations, the team goals and consequences are unclear and often 
conflicting, many of the resources needed to deal with the problems are not available, and the team 
members frequently disagree and argue. 

Employees may be required to 

• identify the focus of team activity and change to a new focus if that will help the team meet its 
goals more effectively. 

• select approaches that show the willingness to give and take direction as needed to further team 
goals (such as coordinate the tasks of various team members so they will serve the larger goals 
of the team). 

• choose approaches that encourage the team to act as a unit and reach agreement when discussing 
specific issues, and identify actions that will help manage differences of opinion among team 
members, moving the team toward its goals while valuing and supporting individual differences 
(manage differences among team members). 
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Locating Information Skill 



The Locating Information skill is an employee’s skill in using information taken from workplace 
graphics such as diagrams, blueprints, floor plans, tables, forms, graphs (including bar charts, pie charts, 
and line graphs), flowcharts, and instrument gauges. Employees are asked to locate, insert, compare, 
and summarize information contained in one or more related graphics. At the lowest level, employees 
are asked to find and insert information in simple graphics. At the highest level, employees are asked 
to make decisions and draw conclusions based on information contained in one or more graphics. 

Four levels of difficulty are described, ranging from Level 3, the least complex, to Level 6, the most 
complex. Examples of the kinds of work-related locating information problems associated with each 
level of the skill are provided. Higher levels of difficulty include the skills described in lower levels. 



Level 3 



Employees must read elementary workplace graphics such as simple order forms, bar graphs, tables, 
flowcharts, maps, instrument gauges, and floor plans. 

Employees are required to 

• find one or two pieces of information in these types of graphics. 

• fill in one or two pieces of information that are missing from these types of graphics (usually 
forms). 



Level 4 



Employees must read straightforward workplace graphics, such as basic order forms, line graphs, 
standard tables, basic diagrams, flowcharts, instrument gauges, and maps. 

Employees are required to . 

* find several pieces of information in these types of graphics. 

* summarize and/or compare information and trends in a single graphic. 

* summarize and/or compare information and trends among more than one workplace graphic, 
such as a bar chart and a table showing related information. 
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Level 5 



Employees must read complicated workplace graphics, such as detailed forms, tables, graphs, diagrams, 
instrument gauges, and maps. 

Employees are required to 

• summarize and/or compare information and trends in a single graphic. 

• summarize and/or compare information and trends among more than one workplace graphic, 
such as a bar chart and a table showing related information. 



Level 6 



Employees must read complex workplace graphics containing large amounts of information and/or 
challenging presentations. These graphics include very detailed graphs, charts, tables, and forms, as well 
as very complicated maps, blueprints, and diagrams. 

Employees are required to 

• make decisions, draw conclusions, and apply information to new situations using one complex 
graphic or several related graphics. 
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Listening Skill 

The Listening skill is an employee’s skill in listening to and conveying work-related spoken information 
in written form. In evaluating the level of the Listening skill required for the tasks of the job, consider 
how important it is for employees to have a complete and accurate understanding of spoken information 
so that the information can be conveyed to a third party. At lower skill levels, employees need to 
understand a few pieces of information. At higher skill levels, employees need to understand all of the 
important information, subtle details, and the correct relationship among the pieces of information (in 
other words, employees must be able to correctly tie together the information given in the spoken 
material). 

Five levels of difficulty are described, ranging from Level 1, the least difficult, to Level 5, the most 
difficult Examples of the kinds of work-related messages associated with each level of the skill are 
provided. Higher levels of difficulty include the skills described in lower levels. 

Level 1 



Employees must understand and convey a little useful information from the spoken material. They must 
understand clues as to the gist of the message or a source of further information. 

Level 2 



Employees must understand and convey some of the important information from the spoken material. 
They may include some incorrect information and they may leave out information the recipient needs 
in order to take action. However, they must correctly understand the sketch of the situation. 

Level 3 



Employees must understand and convey most of the important information from the spoken material, 
as well as correctly report the relationship among the important pieces of information. They may miss 
one or two important pieces of information, but the information present is correct In addition, 
employees must also understand enough information for someone to take appropriate action without 
getting more information. 

Level 4 



Employees must understand and convey all the important information as well as correctly report the 
relationship among the pieces of information in the spoken material. They may miss subtle details or 
may have incorrect supportive information that does not interfere with the main idea. 

Level 5 



Employees must understand and convey all the information from the spoken material (important 
information as well as subtle details). This includes correctly reporting the relationship among the pieces 
of information. 
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Writing Skill 



The Writing skill is an employee’s skill in writing work-related information. In evaluating the level of 
the Writing skill necessary for the tasks of the job, consider 

• the importance of writing mechanics (including grammar, punctuation, and spelling), 

• writing style (Le., smooth and flowing rather than choppy), and 

• professional tone (as defined by the lack of slurs, obscenities, and discriminatory terms) in an 
employee’s written message. 

Five levels of difficulty are described, ranging from Level 1, the least complex, to Level 5, the most 
complex. Examples of the kinds of work-related messages associated with each level of the skill are 
provided. Higher levels of difficulty include the skills described in lower levels. 

Level 1 



Employees’ writing does not convey information adequately because of an overall lack of proper 

sentence 

structure. 

Level 2 



Employees’ writing conveys information adequately. However, there are many mechanical errors which 
interfere with understanding the meaning. Writing may also contain slang and may have weak sentence 
structure. 

Level 3 



Employees’ writing must convey information clearly. Most of the sentences in the messages are 
complete. There are some mechanical errors which do not interfere with understanding the meaning. 

Level 4 

Employees’ writing conveys information clearly. All of the sentences in the writing are complete, 
although they may not be smooth and polished. Writing does not contain any slang. There may be a 
few minor mechanical errors, but these errors do not interfere with understanding the meaning. 

Level 5 



Employees’ writing conveys information clearly. Writing does not contain any mechanical errors or 
slang. Writing has good sentence structure; a smooth, polished, and logical style; and precise language. 
In addition, messages represent the company in a professional manner. 
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